Wisdom of crowds versus wisdom of linguists – measuring the semantic relatedness of words

Wisdom of crowds versus wisdom of linguists - measuring the semantic relatedness of words

In this article, we present a comprehensive study aimed at computing semantic relatedness of word pairs. We analyze the performance of a large number of semantic relatedness measures proposed in the literature with respect to different experimental conditions, such as (i) the datasets employed, (ii) the language (English or German), (iii) the underlying knowledge source, and (iv) the evaluation...

The Multidimensional Wisdom of Crowds

Distributing labeling tasks among hundreds or thousands of annotators is an increasingly important method for annotating large datasets. We present a method for estimating the underlying value (e.g. the class) of each image from (noisy) annotations provided by multiple annotators. Our method is based on a model of the image formation and annotation process. Each image has different characterist...

Evaluating the Wisdom of Crowds

Estimating and forecasting are difficult tasks. This is true whether the activity requires the determination of uncertain future event outcomes, or whether the estimation effort is complex in itself and based on insufficient information. Consequently, such tasks are frequently assigned to experts. Surprisingly, recent research suggests that collectives of non-experts can outperform individual e...

Harnessing the Wisdom of Crowds*

We empirically examine the impact of herding on the accuracy of consensus earnings forecasts from a crowd-based platform (Estimize.com). We monitor user information set by tracking user viewing activities and running experiments. We find that the more public information a user views, the less weight she puts on her private information. While this improves the accuracy of individual forecasts, i...

The “Wisdom of Crowds” Effect